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Abstract The explosion of cloud services on the Inter¬ 
net brings new challenges in service discovery and se¬ 
lection. Particularly, the demand for efficient quality-of- 
service (QoS) evaluation is becoming urgently strong. 
To address this issue, this paper proposes neighborhood- 
based approach for QoS prediction of cloud services by 
taking advantages of collaborative intelligence. Differ¬ 
ent from heuristic collaborative filtering and matrix fac¬ 
torization, we define a formal neighborhood-based pre¬ 
diction framework which allows an efficient global op¬ 
timization scheme, and then exploit different baseline 
estimate component to improve predictive performance. 
To validate the proposed methods, a large-scale QoS- 
specific dataset which consists of invocation records from 
339 service users on 5,825 web services on a world¬ 
scale distributed network is used. Experimental results 
demonstrate that the learned neighborhood-based mod¬ 
els can overcome existing difficulties of heuristic collab¬ 
orative filtering methods and achieve superior perfor¬ 
mance than state-of-the-art prediction methods. 
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1 Introduction 

Cloud computing provides users with transparent and 
elastic infrastructure to deploy various applications (e.g. 
e-mail, multimedia, social network, e-commerce appli¬ 
cations) and allow ubiquitous network to access these 
applications. From a service-oriented architecture per¬ 
spective [2], cloud services refer to those applications 
provided to users on demand via the Internet from a 
cloud computing provider’s servers. The explosion of 
cloud services on the Internet brings new challenges in 
service discovery and selection. To address these chal¬ 
lenges, a range of studies have been carried out to de¬ 
velop advanced techniques that will assist users to choose 
appropriate services m- 

Existing techniques, such as web service retrieval 
[H] and semantic service discovery [IS], can be used 
to locate services satisfying users’ required functional¬ 
ity. Nevertheless, as confronted with a number of func¬ 
tionally similar cloud services, a user may feel hard 
to judge what is the extent of candidate services in 
line with the individual needs. A further comparison 
of the non-functionality of candidate services (gener¬ 
ally the properties of quality of services) is necessary 
in order to make the best choice. To obtain accurate 
and personalized client-side quality values for individ¬ 
ual users, client-side service evaluations jSJinillD] are 
usually needed. However, due to constraints on time, 
costs and other factors |30|, service providers cannot 
dispose a large number of software sensors in cloud envi¬ 
ronments to monitor QoS information for every service. 
Also, it is not realistic for users to carry out large-scale 
testings to experience the individual differences of QoS. 
Consequently, how to obtain personalized QoS of cloud 
services and assist users selecting appropriate services 
become one urgent issue. 
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Recently, researchers from academia and industry 
begin to pursuit solution for this problem by drawing 
lessons from the recommender systems [niiiHiiisiiiisi 

. The main idea of their works 
is to analyze the QoS usage data of users in service- 
oriented systems, further exploit collaborative intelli¬ 
gence [7] to prediction unknown QoS value [niiiHiiiii- 
For example, given a target service i and an active user 
u, the historical quality records of the target service 
i have being observed by the top-k similar neighbors 
of the current user u can be synthesized and taken as 
the predicted value of u to i m- With distinguishable 
quality values of candidate services, users can take a de¬ 
cision on choosing appropriate services. In such a man¬ 
ner, it can avoid direct QoS measurement, and thereby 
save time and economic costs for both service providers 
and users. 

With respect to the collaborative QoS prediction, 
the commonly used methods are neighborhood-based 
collaborative filtering (CF) [1] and matrix-factorization 
[T2| . The advantages of neighborhood-based CF are sim¬ 
plicity, justifiability and efficiency [5]. However, these 
models are not justified by a formal model. Moreover, 
heterogenous similarity metrics and sparsity-sensitive 
problem make these models not robust and scalable 
enough. In contrast, matrix-factorization approaches com¬ 
prise an alternative approach to CF with the more holis¬ 
tic goal to uncover latent features from user-service us¬ 
age data. Thus can alleviate the problem of sensitiv¬ 
ity to sparse data. Since matrix-factorization can be 
presented as a formal optimization problem and solved 
by machine learning methods, thus it provides attrac¬ 
tive accuracy and scalability for QoS prediction. How¬ 
ever, matrix-factorization is always uncertain, resulting 
in difficulty as explain the predictions for users. It is 
very important that recommender systems provide ex¬ 
planations for their recommendations so that users can 
consider and trust them [4]. By the same token, users 
of cloud services certainly hope to get reasonable expla¬ 
nations for the QoS predictions provided by a service 
recommendation system. 

Consequently, a spontaneous concern may be raised 
whether we can develop more accurate neighborhood 
models which overcome existing difficulties, and achieve 
more accurate prediction than matrix factorization. It 
would be a better solution for the task of QoS predic¬ 
tion. Inspired by this, we propose learning neighborhood- 
based models for personalized QoS prediction of cloud 
services. We define formal neighborhood models which 
permit an efficient global optimization scheme and ex¬ 
ploit different baseline estimate components to improve 
prediction performance. Experimental results show that 
learning neighborhood model can overcome existing dif¬ 


ficulties, and perform superior to the-state-of-art pre¬ 
diction methods. 

The remainder of the paper is organized as follows. 
We review some existing works that are most relevant 
to ours in Section 2. We give details of the proposed 
neighborhood-based prediction approaches in Section 3. 
We measure the effectiveness of the proposed methods 
via a set of experiments on real QoS data in Section 4 
and conclude in Section 5. 


2 Related works 

2.1 QoS prediction based on neighborhood model 

The most common approach to CF-based QoS predic¬ 
tion is neighborhood-based models which carry on sim¬ 
ilar users or services to predict the QoS of target ser¬ 
vice. It can be divided into user-based and service-based 
collaborative prediction model. One basic user-specific 
model can be formulated as following, 

.. X/uGTVA'' ^uviXvi ~ 

Tui = fJ'U H yS 5 (1) 

where, A/j^ is the set of k-nearest neighbors of user u, 
Hu and Hv are the average QoS value observed by user 
u and user v. Suv is the similarity between u and v 
and can be described with numerous metrics, such as 
pearson correlation coefficient (PCC), cosine similarity, 
and mean squared difference [S] . To implement service- 
based prediction model, we can just switch the roles of 
users and services. 

Shao et al. m at first proposed the use of collabora¬ 
tive prediction method based on the user. Firstly, PCC 
measurement is utilized to calculate pairwise-similarity 
among all users on the user-service matrix of QoS data. 
Secondly, historical quality values of target service pro¬ 
vided by the top-k similar users of active user are fused 
to achieve prediction result. Follow-up research works 
basically follow this idea but concentrate on improving 
the similarity metrics to accurately quantify the cor¬ 
relations of users or services. Zheng et al. mm pro¬ 
posed a mixed model that integrated user-based and 
item-based approaches linearly by confidence weights 
and proved the mixed model is better than a single 
one. Sun et al. considered the distribution characteris¬ 
tics of QoS data to calculate the similarity [H] . Chen et 
al. [3] and Wu et al. [53] proposed location-aware sim¬ 
ilarity metrics to find neighbors of users and services. 
These neighborhood-based methods do not exploit ma¬ 
chine learning, hence, can be seen as heuristic-based 
prediction approaches. 
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Neighborhood-based methods became very popular 
because they are relatively simple to implement and 
provide intuitive explanations for the prediction results. 
However, some concerns about neighborhood-based meth¬ 
ods always exist. Most notably, these methods are not 
justified by a formal model. The selection of heteroge¬ 
nous similarity metric clearly affects the accuracy of 
QoS prediction, thus make these models not robust. 
When the QoS data are sparse, the predictive power can 
be greatly reduced, resulting in the sparsity-sensitive 
problem. This motivates us to develop more accurate 
neighborhood models to resolve existing difficulties. 


2.2 QoS prediction based on matrix factorization 

Different from collaborative filtering, matrix factoriza¬ 
tion approaches have attractive accuracy and scalability 
thus recently become popular in recommender systems 
m- A typical model associates each user u with an 
user-factors vector pu G , and each item i with an 
item-factors vector qi G . The prediction is done by 
taking an inner product, i.e., fui = PuQi- To exploit ma¬ 
trix factorization for QoS prediction, Zheng et al. m 
[30] using probabilistic matrix factorization (PMF [16]) 
approach to decompose the QoS matrix. For identifying 
latent factors, p’s and g’s, a least-square’s problem like 
Eqj^ are built and solved using gradient descent m ■ 

min V {rm - plqif + XuWPuW^ + Xy\\qt\\'^ (2) 

(n,2)GE 

Matrix factorization can partially alleviate sparity- 
sensitive problem of collaborative filtering. Thus im¬ 
prove the accuracy of QoS prediction. For the last two 
years, numerous efforts have been made on improving 
MF-based models. These works concentrate on utilizing 
of additional information, such as spatial and temporal 
information associated with users or services. Zhang et 
al. |25j use collective matrix factorization that simul¬ 
taneously factor the user-service quality matrix, ser¬ 
vice category and location context matrices. Zhang et 
al. [26] factorize user-service-time matrix of QoS us¬ 
ing non-negative tensor factorization with time infor¬ 
mation. Yin et al. [53] develop a location-based regu¬ 
larization framework for PMF prediction model. Lo et 
al [H] exploit PMF prediction model with a localtion- 
based pre-filtering stage on QoS matrix. He et al. [9] de¬ 
velop location-based hierarchical matrix factorization. 
Yu et al. |24j experience trace-norm regularized matrix 
factorization. 

Matrix factorization is still uncertain, resulting in 
difficulty as explain the predictions for users. Also, data 
sparsity has a negative impact on these methods, as 


data becomes extremely sparse, the prediction perfor¬ 
mance will be not optimistic. 


3 Learning Neighborhood-based Model for QoS 
Prediction 

3.1 Problem Description 

To cope with existing drawbacks of CF-based predic¬ 
tion methods, we suggest using machine learning tech¬ 
niques to build neighborhood model for QoS predic¬ 
tion. New models allow an efficient global optimization 
scheme and exploit different baseline estimate compo¬ 
nent to improve prediction accuracy. We reserve special 
indexing letters for distinguishing users from service: 
for users m, v, and for services f, j. A QoS indi¬ 
cates the observed quality of user u on service i. We 
distinguish predicted quality from known ones, by us¬ 
ing the notation for the predicted value of The 
(u, i) pairs for which r^i is known are stored in the set 
E = {(u ,is known}. Usually the vast major¬ 
ity of QoS values are unknown. To combat overfitting 
in learning prediction model on the sparse data, models 
are regularized so estimates are shrunk towards baseline 
defaults. 


3.2 Neighborhood models 


User-oriented methods estimate unknown quality based 
on recorded QoS of like minded users. Analogously, 
in service-oriented methods, a QoS is estimated using 
known QoS made by the same user on similar services. 
In cloud computing, the context with users is more 
complicated and dynamic than that of services. Pre¬ 
diction leveraged by similar users other than services 
is more reasonable. Thus, our focus is on user-oriented 
approaches, but parallel techniques can be developed 
in a service-oriented fashion, by switching the roles of 
users and services. 

We exploit the neighborhood model proposed in col¬ 
laborative filtering research which allows an effi¬ 
cient global optimization scheme and offers improved 
accuracy. To facilitate global optimization, we would 
wish to abandon such user-specific weights (see SyV in 
Eq0 in favor of global weights independent of a specific 
user. The weight from user v to user u is denoted by 
Wuv are able to be learned from the data through opti¬ 
mization. By this, we can overcome the weaknesses with 
existing neighborhood-based models. An initial sketch 
of the model describes each quality score by Eq[^ 


— ^ui T ^ ^ {'^vi 


vgM, 


(3) 
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where Mu is the neighbor set of user u, bui is the basic 
estimate that we will gradually construct considering 
different factors. 

With respect to the interpretation of weights, usu¬ 
ally they represent interpolation coefficients relating un¬ 
known quality score to the existing ones in a tradi¬ 
tional neighborhood model (recall Suv in EqQ- Here, 
we adopt them in a different viewpoint that weights rep¬ 
resent offsets to basic estimates and residual, — byi, 
is viewed as the coefficients multiplying those offsets. 
For two similar users u and v, Wuv is always expected 
to get high, and verse visa. So, our estimate will not 
deviate much from the basic estimate by a user v that 
accessed i just as expected {vyi — byi is around zero), 
or by a user v that is not known to be predictive on u 
(wyy is close to zero). 

Generally, we can take all users in Mu other than 
u, however, this would increase the number of weights 
to be estimated. In order to reduce complexity of the 
model, we suggest pruning parameters corresponding to 
unlikely user-user relations. Let My be the set of k users 
most similar u, as determined by the similarity measure 
Suy (see Eqj^. Further, we let M^^.y^ = Mi f^My, where 
Mi is the set of users have used the service i. Now, when 
predicting according to formula it is expected 
that the most influential weights will be associated with 
users similar to u. Hence, we replace Eq|^ with; 

'^ui — bui F 1 *^( 2 ;^) I ^ ^ ^ byi)Wuy (4) 

When k = 00 , rule Q coincides with When k = 0, 
fui = bui - However, for other values of k, it offers the po¬ 
tential to significantly decrease the number of variables 
involved. This final prediction rule permits fast online 
prediction, since more computational works, such as 
similarity calculation and parameter estimation, have 
been made at the pre-processing stage. Recall that un¬ 
like matrix factorization, the neighborhood models al¬ 
low a direct explanation of their predictions, and do not 
require re-training the model for handling new services. 

3.3 Components for estimating bui 

In this section, we will gradually construct the compo¬ 
nent to estimate bui through an ongoing refinement of 
our formulations. 

Baseline estimate. Typical QoS data exhibit large user 
and service effects-i.e., systematic tendencies for some 
users to achieve better QoS than others, and for some 
services to receive better QoS than others. It is usual 
to adjust the QoS data by accounting for these effects. 


which we encapsulate within the baseline estimates. De¬ 
note by ^ the average QoS value observed in the entire 
dataset. A baseline estimate for an unknown QoS 
is denoted by bui and accounts for the user and service 
effects: 

byi P- F by F bi (5) 

The parameters by and bi indicate the observed devi¬ 
ations of user u and service *, respectively, from the 
average. For instance, suppose that we want to find out 
the baseline estimate for the response time of Google- 
Search service by Tom. Now, presume that the average 
response-time, /i = 2ms. Furthermore, google-search is 
recognized better than an average search engine ser¬ 
vice, so we can suppose it is faster 0.5ms than the av¬ 
erage. On the other hand, Tom’s network condition is 
not a good, which tends to be 1ms delay than the av¬ 
erage. Thus, the baseline estimate for Google-Search’s 
response-time by Tom would be 2.5ms by calculating: 
2-0.5 f 1. Substituting Eqj^into Eq|^ we will obtain the 
first neighborhood model (NbModell). To estimate by, 
bi and Wyy one can solve the least squares problem: 

min ^ [rui - bui - \M('i.y^\~^ ^ (ryz - by^)wuyf 

F Al ^ wly + X2{bl + bf) 

( 6 ) 

Here, the hrst term Yl,(u i)GE(-)^ strives to hnd 6„’s, bM 
and WuyS that fit the given response data. The second 
part and the third part both are the regularizing terms, 
used to avoid overfitting by penalizing the magnitudes 
of the parameters. Ai and A 2 are the specific regular¬ 
ization parameters 

Weighted features. The baseline estimate just considers 
the mean effect of user and item in an intuitive manner. 
However, more bias information (such as user features, 
item features and time bias information, etc) can be uti¬ 
lized to enhance the prediction model. We may consider 
an unknown quality value Vyi as a linear combination 
of the features of user and item. Here, the user-specific 
QoS mean and the service-specihc QoS mean are taken 
as the two key features, because that this two are strong 
features to reflect bias effect of users and services, and 
the expect prediction of QoS can be found in a value 
domain determined by them. Denote by /i„ the average 
QoS value observed by user u and /ii the average QoS 
value observed by service i. A feature-weighted estimate 
for is denoted by byi as followings: 

bui = WutJ.u+ WifXi 


(7) 
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(c) QoS Prediction Using NbModelS 
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(b) The Learned Parameters of NbModeB 


Fig. 1 An example on QoS prediction with learning neighborhood-based prediction model. 


The parameters Wu and Wi indicate the feature impor¬ 
tance of user u and item i, respectively. Substituting 
Eqj^into Eq|^ we will obtain the second neighborhood 
model (NbModel2). In order to estimate Wu, Wi, we 
need to solve the following least squares problem: 

min ^ [rui - bui - X! 

+ Al ^ Wyy + X3{Wy + Wf) 

( 8 ) 

Here, regularizing term, X 3 {Wy + wf), avoid overhtting 
by penalizing the magnitudes of the parameters Wy’s 
and Wi’s. 

Hybrid approach. Beyond estimating bui based on ei¬ 
ther the baseline estimate or the weighted features, we 
may combine them together to have both of worlds. 
This leads to a new prediction rule for bui- 

bui = fJ- + by + bi + WuPu + WiHi (9) 

Substituting Eqj^ into Eq|^ we will obtain the third 
neighborhood model (NbModelS). To estimate Wu, Wi, 
by, bi, we need to solve the regularized least squares 
problem as followings: 

min ^ [’’m - X! - byi)wyy]‘^ 

+ Al ^ w1_y + X2{b1^ + b1) + X3{Wy + wi) 

( 10 ) 

3.4 Models Learning 

In a sense, our neighborhood models provide two-tier 
models for personalized QoS prediction. The first tier. 


byi, describes general properties of the service and the 
user, without accounting for any involved interactions. 

The second tie-” Neighborhood tier” contributes fine grained 
adjustments that are hard to profile. Model parameters 
are determined by minimizing the associated regular¬ 
ized squared error function through gradient descent. 
Recall that — fyi. We loop over all known rat¬ 

ings in £. Eor a given training case r„j, we modify the 
parameters by moving in the opposite direction of the 
gradient, yielding: 

by by -b X2by) 

- bi -i— bi + 7i(e„i — X2bi) 

- Wy^ Wy+ 7i(e™/r„ - XsWy) 

- Wi ^ Wi + ^i{eyiPi - X^Wi) 

^uv '^uv T 2 ^ 1 *^ [i'tu) I ^ 

Note that, update rules set forth can fit all of the least 
squares problems in Eqsj^ refeq :8 and When as¬ 
sessing the method on a given dataset, we took ad¬ 
vantage of following values for the meta parameters: 

Al = A 2 = A 3 = 0.001, 7 i = 72 = 0.001. It is benefi¬ 
cial to decrease step sizes (the 7 ’s) by a factor of 0.9 
after each iteration. Another important parameter is fc, 
which controls the neighborhood size. Our experience 
shows that increasing k always benefits the RMSE of 
the results on the test set. Hence, the choice of k should 
reflect a tradeoff between prediction accuracy and com¬ 
putational cost. A toy-example on Qos prediction using 
NbModelS is given in Eigurej^ where we let fc = 5. 

4 Experiments 

4.1 Datasets 

To evaluate the QoS prediction performance, we use a 
large-scale dataset collected by Zheng et al. [5D1ITT] . The 
dataset consists of a total of 1, 974, 675 real-world web 
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Table 1 Performance comparisons of QoS prediction models using different matrix density, where and ’**’ indicate Ist-class 
and 2nd-class, respectively. 


Methods 

MD 

=0.5% 

MD= 

=1% 

MD= 

=5% 

MD= 

10% 

MAE 

RMSE 

MAE 

RMSE 

MAE 

RMSE 

MAE 

RMSE 

GMEAN 

0.9721 

1.9735 

0.9915 

1.9726 

0.9860 

1.9740 

0.9920 

1.9725 

UMEAN 

0.8873 

1.8914 

0.8899 

1.8705 

0.8746 

1.8600 

0.8744 

1.8575 

IMEAN 

0.8317 

1.9326 

0.7870 

1.8136 

0.7002 

1.5746 

0.6890 

1.5410 

UPCC 

0.9709 

1.9727 

0.9468 

1.9457 

0.6173 

1.3925** 

0.5446 

1.3119 

IPCC 

0.9721 

1.9735 

0.9888 

1.9716 

0.6675 

1.4272 

0.6430 

1.3798 

UIPCC 

0.9708 

1.9725 

0.9453 

1.9430 

0.6162 

1.3900** 

0.5439 

1.3102 

PMF 

0.8317 

2.0624 

0.8195 

2.0010 

0.6354 

1.5071 

0.5541 

1.3393 

NMF 

0.7656* 

1.8480 

0.7092* 

1.7650 

0.6516 

1.5145 

0.6461 

1.4527 

BiasedMF 

0.8052 

1.7643 

0.7832 

1.7128 

0.6376 

1.4361 

0.5518 

1.3088 

NbModell 

0.7986 

1.6989* 

0.7471 

1.6163* 

0.6106 

1.5333 

0.5336 

1.3563 

NbModel2 

0.7678* 

1.7942 

0.7196** 

1.6436** 

0.5817** 

1.3943** 

0.5156** 

1.2832** 

NbModeia 

0.7838** 

1.7847** 

0.7428 

1.6582 

0.5793* 

1.3795* 

0.5138* 

1.2739* 


service invocation results are collected from 339 users 
on 5, 825 real-world web services via PlanetLab plat- 
form[^ which is a global research network that supports 
the development of new network services. This dataset 
can be treated as a set of usage data for real-world cloud 
services from distributed locations. For more details, 
users can refer to works mm- In our experiments, we 
only think the response time (the range scale is 020s 
). However, the proposed approach can be applied to 
other QoS properties easily. 

4.2 Evaluation metrics 

Mean absolute error (MAE) and root mean squared 
error (RMSE) metrics, two basic statistical accuracy 
metrics m, have been extensively used in performance 
evaluation of rating predictions [1] , are used to measure 
the QoS prediction performance of selected methods. 
MAE is defined as 

y^... i \^ui '^ui\ 

MAE = —' ( 11 ) 

where r^i is the observed QoS value, is the pre¬ 
dicted one, and N is the number of test cases. The 
MAE measures the average magnitude of the errors in 
a set of forecasts, without considering their direction 
nnj. RMSE is defined as 

ruse = ( 12 ) 

The RMSE is a quadratic scoring rule which measures 
the average magnitude of the error. 

Both MAE and RMSE, a smaller value indicates a 
better prediction accuracy. The MAE is a linear score 
which means all the individual differences are weighted 

^ http://planet-lab.org/ 


equally in the average. Since the errors are squared be¬ 
fore they are averaged, the RMSE gives a relatively high 
weight to large errors. This means the RMSE is most 
useful when large errors are particularly undesirable. 

4.3 Comparison 

To show the prediction accuracy of our neighborhood- 
based approaches, we compare our methods with the 
three kinds of popular approaches: 

1. Statistical approaches 

(a) GMEAN takes the average QoS value of whole 
dataset as the predictive QoS value of user u to 
service i, i.e. = /t; 

(b) UMEAN takes the average QoS value known by 
u as the predictive QoS value of m to i , i.e., 

^ui - 

(c) IMEAN takes the average QoS value observed 
from i as the predictive QoS value of u to i, i.e. 

'^ui — 

2. Heurstic-based CF 

(a) UPCC is user-based collaborative prediction model. 
Top-k neighbors of users are found using PCC- 
based similarity [HI; 

(b) IPCC is item-based collaborative prediction model. 
Top-k neighbors of items (services) are found us¬ 
ing PCC-based similarity [5S]; 

(c) UIPCC combines the user-based and item-based 
collaborative prediction approaches and employs 
both the similar users and similar services for the 
QoS value prediction [29]. 

3. MF-based approaches 

(a) PMF uses probabilistic matrix factorization [TB] 
to factorize user-service QoS matrix for the pre¬ 
diction |5D] : 

(b) NMF uses non-negative matrix factorization [T3] 
to factorize the QoS matrix into two matrices p 
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and q, with the property that all three matrices 
have no negative elements; 

(c) BiasedMF exploits a combination of baseline es¬ 
timate (same to Eqj^ and matrix factorization 
prediction rule for collaborative filtering [T^] . We 
adopt it for the QoS prediction. 

For the memory-based CF methods, we chose the 
neighborhood size of users at A: = 10 and services at 
k = 50. For the matrix-factorization based methods, 
the regularization parameters for user and service are 
set at A„ = A„ = 0.001, and the dimensionality of la¬ 
tent factors is fixed at 10. For all selected methods, we 
use their implementations in LibRec which provides 
a Java library for recommender systems and a chance 
for reproduction of experimental results. Note that, for 
a fair comparison of selected methods, all of them ex¬ 
ploit only the information supplied by the user-service 
QoS matrix, and no additional information (e.g., geo¬ 
locations of users) is allowed (we consider to extend the 
proposed methods to exploit additional information in 
the future). 

In the real-world, a user usually only invokes a few 
cloud services. Thus leading user-item matrices of QoS 
data sparse. This can be always observed in the rec¬ 
ommender systems DEI- For instance, the density for 
the well-known MovielenslM dataset |3 is 4.5%. The 
data density for Netflix Prize is about 1.4%. To re¬ 
flect this natural phenomenon in experiments, we ran¬ 
domly remove entries from the user-item matrix with 
different density, specially, we take 0.5% — 1% percent 
for the case of sparse data and 5% — 10% percent for 
the case of dense data. For instance, MD (Matrix Den- 
sity)=0.5% means that we randomly select 0.5 percent 
of the QoS entries to predict the remaining 99.5 per¬ 
cent of QoS entries. The original QoS values of the re¬ 
moved entries are used as the expected values to study 
the prediction accuracy. The above-mentioned methods 
and neighorhood-based variants, NbModell, NbModel2 
and NbModelS, are employed to forecast the QoS val¬ 
ues of the removed entries. The experimental results are 
shown in Table [T] 

Depending on Table NbModelS and NbModel2 
respectively rank the first class and the second class on 
both MAE and RMSE in the case of dense data. Both 
models obtain smaller MAE and RMSE values consis¬ 
tently for response-time with MD=5% and MD=10%. 
MAE and RMSE values of neighborhood models be¬ 
come smaller, since denser matrix provides more infor¬ 
mation for the missing value prediction. In the case of 
sparse data, NMF achieve best performance in term of 

^ http://www.librec.net/index.html 
® http://grouplens.org/datasets/movielens/ 

^ https://en.wikipedia.org/wiki/Netflix_prize 


MAE followed by three neighborhood models. However, 
our methods perform much better than all other coun¬ 
terparts on RMSE. Among all the prediction models, 
our methods achieve better performance on both MAE 
and RMSE, telling that learning neighborhood model 
can achieve higher prediction accuracy. Also, neighborhood- 
based models preserve the explainability of memory- 
based CF, and enable to give users a reason for their 
predictions. 


4.4 Impact of Top-K 

To examine the impact of top-k neighbors selection on 
neighborhood-based prediction models, we distinguish 
from two cases: sparse data and dense data. With sparse 
data, we found that increasing k value cannot result in 
significant performance improvements and sometimes 
we may experience decreased performance. The exper¬ 
imental results are shown in Table i where fui = Ki if 
fc = 0. We believe that there are two reasons for this. On 
one hand, sparser matrix cannot offer the ’’neighbor¬ 
hood tier” more information to contribute fine grained 
adjustments. On the other hand, the leaned component 
bui (sees EqQ has given ideal predictions. Nevertheless, 
’’neighborhood ties” can be used as a regularization 
component to avoid the overfitting of baseline predictor 
even if the usage data are sparse. 


Table 2 Performance of neighbordhood-based models in case 
of sparse data. 


Methods 

k 

MD= 

=0.5% 

MD 

=1% 


MAE 

RMSE 

MAE 

RMSE 

NbModell 

0 

0.7987 

1.6990 

0.7411 

1.5770 

80 

0.7986 

1.6989 

0.7471 

1.6163 

NbModel2 

0 

0.7678 

1.7942 

0.7428 

1.6567 

80 

0.7678 

1.7942 

0.7196 

1.6436 

NbModelS 

0 

0.7840 

1.7848 

0.7409 

1.6474 

80 

0.7838 

1.7847 

0.7428 

1.6582 


In the case of condensed data, we conduct experi¬ 
ments to see the impact of top-K similar users based on 
NbModel2 and NbModelS, as both of them are more 
robust than NbModell. The experimental results are 
shown in Figure From Figure we find that the 
RMSE of both models consistently decrease as increas¬ 
ing the value of k with different matrix density (range 
from 5% to 15%). While the value distribution of MAE 
presents U-shaped curve, and the best configuration for 
our dataset is about k = 80. In addition, more gains 
can be noted when the matrix becomes denser as for 
neighborhood-based models. For neighborhood models, 
since the computational cost always increases with the 
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NbMode2 



(a) 


NbModeS 



(c) 

Fig. 2 Performance of NbModel2 and NbModelS with different 

increment of K, the choice of k should reflect a tradeoff 
between prediction accuracy and computational cost. 


5 Conclusion and future works 

Based on principles of collaborative filtering and ma¬ 
chine learning, we propose a neighborhood-based frame¬ 
work for making personalized QoS prediction of cloud 
services. The framework provides an efficient global op¬ 
timization scheme, thus offers robust and accurate pre¬ 
diction results. Also, it preserves explainability for QoS- 
prediction tasks which would be helpful for users make 
more definite selections. The extensive experimental anal¬ 
ysis indicates the effectiveness of our approach. 

The main contribution of this paper lies in two¬ 
folds. First, we suggest learning neighborhood-based 
formal prediction model for personalized QoS evalu¬ 
ation of cloud services, and provide three distinctive 
variants of the model. Second, we prove that learning 
for neighborhood-based QoS prediction provides more 
robust and scalable method compared with traditional 
memory-based CF and matrix factorization techniques. 


NbMode2 



(b) 


NbModeS 



(d) 


k and matrix density. 

Since neighborhood-based models are distinctly of 
matrix-factorization, we would like to integrate them 
together to have both of worlds in the future. Result¬ 
ing from the lack of real-world datasets for conducting 
experiments, we only conduct experimental studies on 
response-time, thus expect to adapt the proposed meth¬ 
ods to the prediction tasks of other cloud service QoS 
properties, such as reliability, throughput and scalabil¬ 
ity. In addition, since we set out a general framework of 
the neighborhood model, extra information, such geo¬ 
information of users and items, temporal use of service 
invocation, can enter into this framework to offer more 
accurate prediction results. 
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